Today, better integration of business information systems throughout the enterprise and incorporating more recent database management technologies results in real managerial competitive advantages. Information to support management decision making typically is extracted from operations systems and loaded into a data warehouse. Information as used herein includes data internal to a corporation that is generated in the day-to-day business operations of the corporation, aggregations and summaries of the internal data, and external information such as demographic and geospatial data that can be related to the internal data. A data warehouse is a large storage facility within the information technology, IT, system that typically holds the current and historical information across an entire enterprise. It most likely consists of numerous databases and may exist on multiple platforms (operating systems). A database can be defined as a collection of information organized in such a way that a computer program can quickly select desired pieces of data. One can think of a database as an electronic filing system.
In the past, it was only viewed as necessary to capture periodic data extracts supplied by operational computing systems to update the data warehouse on a weekly or monthly basis. Strategic dependence on data warehousing was viewed as long-term and historical, not as a real time activity. In contrast, today's business environment is worldwide and does not stop at five o'clock P.M. For example, the Internet allows commerce to continue 24 hours a day, seven days a week. As a result, customer service support and normal telephony operations within many business organizations likewise operate around the clock.
Furthermore, marketing practices have evolved from mass media advertising to more personalized target marketing, which cuts expenses while providing customized treatment. To enable customized marketing, companies must know and understand their customers. Much of this understanding is made possible through groups such as Decision Support Services (DSS), the business group within an enterprise that has traditionally supported data warehousing. To be effective in supporting the extended operating and marketing efforts of an enterprise, information should be processed and pushed back from DSS into the customer contact areas such as customer service support, telemarketing and advertising campaign management as rapidly as possible. Thus, it is preferred that DSS be capable of receiving, processing, and providing information on an ongoing, near real time basis.
This disclosure, referred to as Distributed ETL, improves on current DSS services by providing many of the advantages of scalability of an extract, transform and load (ETL) application and the near real time event processing of an enterprise application integration (EAI) application while reducing or eliminating some of the disadvantages associated with each.